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Vv^BSTEACT r ' ' 

\ Phase I of the Measurement, of Respondent Bu 

.described* The focus of the study was upon correlates of 
perceived respondent burden, i.ea^ factors associated with 
pr«t5^ence of negative feelings such as annoyance, frustratil) 
inconvenience which may be experienced by survey pafticipaa 
face-to-f ace- interviews at their homes^j. The effect of inter 
(25- and 75-minute treatiaents) and respondent effort upon r 
burden^^'Vas assessed. Interviews were conducted- with 500 coa 
persons i^^ousehoi^s- located in a predominately white midi 
suburbs Hosr\ questions sought factual information, rather t 
opinion or attitiide data. Results indicated: (1) no relatio 
found- between interview "length (/iL) and overall item nonres 
XL was associated with willingness to be re-interviewed Hit 
percent more of the short interview subjects willing to ali 
interviewer to return next year; (3) IL affected attitudes 
the interview, by eliciting more negative responses to quest 
asked specifically about length: and (U) generally, iricreas 
effort required to complete the respondent s task (retrieva 
records) had no effect, on behavioral' or attitudinal indicat 
respondent burden, (Author/^RL) 
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... -.CKGROUND OF THE rTUO^ 
Jntroducti on 

e . 

This paper :e5cr::es Phas I of the Measuremen . 3f f ::ndent 
Burden, an experimental study cor ucted by the Bureau .rv So: Science 
Research for the Department cf Hcusing and Urban Development. The 
focus of the study is upon ccrre ates of .gel f perceived respondent burden', 
^ that is factors associated v/ith z'ne pr.eseoce of nega :. : ve feelings such 
as annoyance, frt^stration . r in : onvenience which may be e isrienced by 



survey participants. The r:tu& '.s limited to survey situ: :ons in which 
face-to-face^|nterviev/s are conducted in respondents' hon^i , - ° 

, The issue addressed by :his research are pf interest to HUD. in 
connection with its sponsorship cf.. the Annual Housing Survey (AHS) , a 
face-to-face household survey wh'ch Js the primary source of information 
about the condition of the nation's hous i ng supply. The level of refusal 
rates in the AHS, reoorts from field obsei*>{ers, and occasional complaints^ 
by individual respondents have caused concern within HUD with the burden 
which the survey may be fmposing^upon AHS respondents, ' 

The issue of respondent burden is also of interest to a wider 
audience of survey end evaluation researchers^ who are concerned With 
maintaining or improving response^rates in their data collection activi- 
ties, and to government personnel charged ;wi th|managi ng or regulating 
Federally-sponsored data co,) i.ect ion- efforts . ^"^e present, research is 
intended to. provide information whichwill be useful to this broader 
research communi ty, including both sponsors and per'formers of education 



research, espe ial y t^a: -esea:-c:- lnvoi\ -oluntary face-to-face 
rata coi lectin from inrl v : :.3]5 : - -ih as :r3T::unity members , parents , 
and student?. 

'lb~ Study Deslcjn 



The tucy 3s being imc erented In tv^r phases. Durlnr the first 
phase, we fc: ed o vest He two ccnf I i c : i nr-'theorl e3 :he conventional 
v/ls'dom'of many layrsn and pra : t : - ioners- : lai 'on^er interv?ev/s and ^ 
questions requiring sor-^ ^^ztzn'^ ■ ort on tr.^ part^'of tne respondent 
wTII I ncrease respondent bu den, .-sus the r Dothesis c' some survey 
-research methodologisrs that ^-c- quest « onr i res and zee nTefforts 

ft 

may actually reduce bcrden errrzrrz lng the ~:portance of the, survey in 
the-eyes^of the respondent, ""c zr— ess this ssue, born I-^terview length 
and respondent effort were — r/ir: -r:ed during the first phase of the 
^survey, and their effecl^ upon r-eimndent burden was assessed. 

lnteT>>iew length was n::De:~£}r iona lly defined as **nurnber of minutes 
of interview time,'' With 2r and ~ aiinute treatments used. The /'effort*' 
variable was a^i'so separatHii ir— tr/.o treatments: Vecall, in which 
respondents were asked ^tc prcv-n.. estimates ba^ed on memory for answers 
to selected expenditure G:j~st : .tttt: and retrieval, in vvh^ich respondents . 
were asl<ed tp consult ch£i::ir.3cj-r r other records to answer these questions 

The lenqth and ef^T^rt vai" 3b!es v/ere itrossed to create four 
treatment groups, each of Which received a different version of the , 
Interview:, a short recall Interview (Group l)-; "a" tong reca.l 1 inlerview 
(Group 2), and short and long retrieval ihterviews (Groups 3 & 't) . 
The total study samp^le*^%e was 500 (see F'igure^^J) . ^ All versions^ of the 
fnterview. instrument were based on the Annual Housing Purvey, and con- 
tained a wide range of items about .fami ly 'compos i t ion, housing charac- 
ter is| I csrenepgy use^ transpqtrt^tion, neighborhood characteristics, etc. 
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R2S"::ndent :i -z.^- was rrteasured by direct ::jest! ~ ^ the 
res:r::dent :hrough 2 : ---adm.i ntstered reactldn f^rm, ' /n /as gi^ r-'^^ 
at ' ne .conclusion of "treatment'' intervievy; (For f t'-- 

res'^-^dents in Grc. p r : 100 in Grou::* 2, the reactior fc — 

♦ 

use: since trhsse resr—ri-snts wi 1 1 be rei nterviewed for : ^ s — . d 

'phas . of the r' udy,) s::r?ondent burden was also assessed in:rir .:t by 

exarruring 1 ~ respor ^stes a"nd other^ behavioral indicar— - or - ne 

various tre: nt grr-.-s . ^ * 

••• • . • 

Ai: Tt2irv?ev.: for Phase 1 were cojiducted in thf; £:::ijrbs c* 

Ph?]adelph ouring _::ruary,* I98O, The study sample cons : ted o a 
total of 75 :u3eholc -lusters; within each cluster al 1 Ph^e 1 treatments 
were used; n ell gi e^-respondent was def.Tned as a household member-who . 
was ''knowk: .r-able azzut household expenditures,'' If households coulJ 
not be COPT :ec afrr^ h attempts, or if the householder refused to be ' 
Jntervtewei the* saTrr: i i ng scheme provided for the selection of a. substitute 
household, ^ , ^ ' - 

o For the second phase of the- study, the effort variable was ^ 

eli^minated, and the-'third mahi pu!ated .vari able — single vs. repeat 
administration of identical questions over time — was introduced. 4 The 
second round of interviews was conducted approximately 10 months after 
the first interview wi th 200 respondents from the Phase I sample. Data 
from this phase are currently being analyzed, and are not reported here. 



i Findings: Phase I 

The sr.:dy flndingi ^unrnarized below should be interpreted it i 
two factdrfs Tnind: 

' . fnterviews ^ere :onducted in households located ?- a 
predom.- ir white middle clas suburb. Therefore* the study findings 
may ncz :^ "^pHcable to crner -oulations> for example, persons Hvlng 
In rur3 hs or in inner ci: or torother types 'of interviev; 
S Itua::: n:::- 

2 IS interview Jea Ith topics of moderate salience for 
most rt^ijpc : ~ts; most quest' sought factual Information, rather than 
opinlcn cr ^ titude data. Tc: results of the study might have been quite 
dSfr^—nt if more (or less) ~£:::-:rondent-per:tinent topics had been discussed, 
or jpin'fc^ or att4tude' qusi tions had:Jb.een emphasized.. 

Differ ences Between Respond£'-:rs and Refusers 

© To achieve 500 inrerviews/^^total of 886 household were. contacted; 
In 184 cases no contact cpu"d b^\estab fished, and ]ri -202 cases, the house- 

.\ '\ : • . ./ ^ . . , \ 

holder refused to be interviewed. No attempt was made in this study to > 



convert initial refusals., but a brief interview was conducted with ^hose 

refusers who cooperated (hN107) . The 202 persons refusing to be Inter- * 

viewed constituted 29 percent of those contacted' for t^he survey. Altl^ough 
/respondents were younger than refusers, no other significant differences 

' /. ■ ■■• ■ •■ ■ ■" • ' • \ ■ 

between the two groups were found in terms of. demographic characteristi^cs, 

. ■ ^ ' ■ ■ ■ 1 ■ 

or .for 'those variabiles which might have affected the 'interview situatioh 

(i.-^f time of day of the attempted interview, or experience of the inter- 

viewer involved). However, refusers and respondents differed in their 



\enera^ attitudes towards the usefulness of survfeys, ar6 !n their past 

participation in intc?rview situations, wi th refusers . 1=:SS likely to rate 

• ' , — 

surveys as cseful., and less likely .to have participate: i- previous sur eys 

\^ '0 V^hile the^t^e-required for an intervJew was f equently the 
reason gtven-for the refusal, the actual length of t i 2 /hen announced 

to-^ehe respondent at the door— i .e. , 25 or 75 minutes — 'd not seem to 

"I ^ ' • ' ' ' ' ^ ' 

contribute in . any appreciable way to the' tendency to -e use.. The per- 
centage of refusals occuring after the specific time W3£ rentioned was 
virtually identical (about 385^0) for both the jonr; ar snort ;,/^erviev 

groups. ^ _ 

^Moreover, tht^ intervjewers' prior knowled-2 of the length c ' 
the interview to be attempted was not related to th 2 overall percentege 
of refusals obtained, which was 2-7 percent. for the short vs. 30 percent 
for the long form..' , 

Gene ral Reactions to the S<«rvey ^ 

Among the 500 persons cohsentihg to be interviewed, reactions 

to the siWey v/ere overv.'helmingly "positive. Only two of these respondents 
■ • ■ ^ ,. ■ ■ ' - ■ 

brol<e. off the interview, pr ior to completion (both of these were in the 

long interview group). Large majorities felt that .the interview was at 

least sdmewhat interesting (87%) ?nd important (88%), and that their 

time and effort were ^t least somewhat well spent (90%). Eighty {.ercent 

saS^f'that they Would be willing to be reinterviewed next year. , ^ 



Rj>1atlonsMp of P^esr■r- dent Burden 

■tn Interview Lerctr- :. Effort , 

J© Table i Hi.-:ninar?zes some df the key findings With respect to ' 
Interview length, effort, a nJj: sel f- reported burden. 

^ No relcticnshlp was found between interview J^engjth and overall 
Ifem oonresponss. On the average, both the long and short interview 
groups failed to answer'-only about three percent of the questions they 
.were asked. r ' '■',[■ 

c^^owever, interview length was associated with willingness to . 
'be re- interviewed with \h percent more'of the short interview subjects 
Indicating that they would be willing to allow the interviewer to return 
next year. - 

o interview lengt h als.o affected attitudes towards the interview, 
• but only to the point of eliciting more negat I vs>esponses to questions 
which 'asked^ specifically about -length. The longer questionnaire was not 
-seen as being more of an overall nuisance (Table 2), or as being more or 
>ess tnterestTng, Import-ant, or difficult than was. the shorter inter- 

■ , ■ ■ ■ ■ 

view. 

• . 9 In general. Increasing the effort ^equ!red to complete' the 
respondent's task (retrieval of records) had no effect on behavioral or- 
attltudinal indicators of respondent burden. For example, there were 
no significant differences between the recall and retrieval groups m 
Item nonresponse rates (whi cK were-2. 89% and 2. 82% respect ively) , in = 
willingness to be reiriterviewed (80% of both groups agreed), nor In the 
proportions of respondents within each group who fel't that the Interview 
was uninteresting, unimportant, not time well spent; etc In short, 
asking respondents to retrieve records neither Created^ a burden (as is ^ 
assumed by some practitioners) nor did it generate a/more positive 
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TABL£ 1 

RESPOHDEMT BURDEN INDICATORS, BY INTERVIEW LENGTH, EFFORT AND TREATMENT 
i ' (in Percenlacjes) 





Lengtli 


. Effort 


Treatment 




Indicators i 


Short 


Long 


Recall 


Retrieval 


Short 
Recall 


Long 
Recall 


Short 
Ret r leva 1 


Long 
Retrieval 


TOTAL 
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». Significance ' » 
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.05 






2. Attltudtnal 






















A. Answers to Questions Concerning Interview 
Length: 
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* • Use of shorter questionnaires '^^/ouId be an 
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35 


57 
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Significance / 
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.01 






• Overall, tine end effort put Into answering 
questions was "not very wel 1 spent" 
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. ■ " Significance 


P 


i:..05 
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■B. Answers to Questions Concerning Other 
Aspects of the Intoirvlow 






















a Overall Interview's '^bot very Interesting" 
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• On the whole, answering the questions t^^as 
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^ „ Significance 




n.s 




n;S 
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.s 






Num^r Responding . I^ -1^7 
Missing Cases . i-8 


136-1^*3 
h-\\ 2-9 




I^1-1M« 
^-7 




67-72 
1-6 


71-73 
1-3 


69-71 (286-290> 
3-5 (6-10) 



'Significance based on chl-square test on collapsed variables, i&.g., too short/about right vs. too long. 
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TABLE 2 



mm THE mimi value" of the survey Smd other mm tasks 



' W t!j8 tc tk, w jr«i all called upon t'o do various things which w» not pertlcularly anjoy, In fact, m of th«!8 te ' 

P!^^..'."j.''!?.^*'^^.«y ^^^^^^^ « w"'P«'"e'' doing othflp tmm tejkt," 



(FOil EACH TAJK Pl/T DOVH 



THE NOHBER OH THE SCALE-i THROUGH lO-tlHICHHEST DESCRIBES HOW HUGH THE TASK BODiEflS YOli.) 



\ " Ml 
' Sother Ho 
. -^tAII 



^12 3 5 S 7 8 9 10 



flothflrj'He 
a Lot 



ShortilecAll 
I $0 



Long Recall 



Shoi't Ratrtcval 
■ X SO 



Long Retrieval 
X ' SD 



.TOTAl I Mtr 
I SO I Reipdlng 



fl./Answerlng the Interviewer's , , ' ' 

• . questions. .during this survey , , ... 2,68 ^ 2,I|0 J.OJ 2,60 ■ ' 2,62 " 2.22 , .2,7'i'". 2,18 2,77 a,]** 
b, 'FI!iln9out tat3xfoTO , . 5.]6, 3.38 6.03 3.62 6.6I| 3,53 M6 ■ 3.37 6.13 ,3.li7 



C, ' Balancing checkbook against • , 

bankstatcipc^t,... 3,86 3,00 " ^3 3.03 kM 3.08. i|.72 3:28 ', 3,10 

d, Answering a public opinion , . i 

survey about which political ' 



0. Going to the polls tp vote an 

election day , , , 2,59 2.50 



2.07 2.2? > 3.16, z. 



f, Getting your car inspected 

th« Jtaie , M2 3.59 ■ kM 3.02 if.i7^J,32 5.12' 3.78 
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10 ■ 



Hissing 
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2,77' 2,3'> 
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(23) 


2.51 2.57 • , 


(27^) 


(22) 


V.63 IM 


m 


(29) 
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reaction by cbnveylng to the respondents 'the sugges\tton that the 
survey was very important, " / 

0 The actual behavior of persons assigned to recall, and re:r?eval 



grouRS did not always conform to instructions; some "recal I'V^resoondents 
chose to check their records, and some .'^etr leva I" respondents did not 
comply w;th this instruction* When effort was examined as a behaviora! 
rather than as a manipulated var iab1e--that Is, v/hen respondents were 
grouped by the percentage of times /they actually checked their records — 
no significant relationshi ps between effort and indicators of burden were 
found. However^ in botH"groups, persons who never referred to their 
records at all expressed negative attitudes more frequently than did 
, Other respondents'. Therefore, there may be some association between 
feelings of burden, unwll 1 Ingness- to exert efforts in t'he Interview 
srtuatIon,'and, therefore, data quality which this particular study was 
not designed to probe fully* 

Interview Length and Respondent Burden , 
w ith Other Factors Considered 

The relationship between respondent burden and Interview length 

was examined with other variables (both attitudinal and demographic) 

held constant. The key "-f I nding emerging .from this analysis was that,/a 

general belief In the efficacy of surveys 'is an important factor influencing 

the extent to -which Interviews of var lous j lengths will be perceived as 

burdensome. That is, among respondents agreeing that '^answering surveys 



Js of direct benefit to the people who anjswer," the relationship between 

'or t 



length and key Indicators of burden is weakened. For example, for this 



group, the percentage refusing to be reinterviewed is generally low, 

no matter if the short (8%) or the long (10%) treatment had been applied. 

in contrast, for respondents who do not agree that surveys -ape beneficial . 

o 

the relationship between length and burden becomes much stronger. For 
this groups, the percent? refusing to be reinterviewed are 19 percent for 
the short and hh percent of the long' Interview groups (a difference of 2S7S) 
(See Table^^3j^^ 

Length. Effort, and Data Quality 

9 Based on the limited measures available in this study, no 
r^^^onship was f^uhcj, between 'interview length and either self-reported 
estimates of data /accuracy V or data completenes^^, as measured by item 
nonresponse. Contrary to common fy held assumptions, item nonresponse 
did not increase during the course of the long interview. Rather,, item* 
nonresponse was a function of question sensitivity, with income questions 
asked at the beginning and end of^t^he--i~fvt-e;;^v4~ev%ueJ^^ a similar 

(relatively high) level of nonresponse. ; 

^ No general relationship between record checking and self- 
reported estimates of data qua 1 1 ty was found; Although, on the average, 
retrieval respondents did re'fer'to records more frequently than those 
asked to rely on memory, differences In perceived accuracy of response ' 
between the two groups, while in the e>^pecte^>di recti on , were ndt 
.statist ical ly significant. Only among persons favorably predisposed 
toward surveys, did a significantly greater proportion of retri^e^al than 

Of recall respondents perceive their responses as-"very accurate.'^ 

V ■ ■ • ... 



TABLE 3 



mmm mm short and long iraviEw groups in willikss to be reinttoieweo hext 

'WHEN A'lTITUDES TOWARDS SURVEYS ARE CONTROLLED 
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Answering survey Is of direct benefit 
to the people who answer: 














* . : \ , . ^ 

Strongly aqrees/aqrees' . , . , , ' 


8 


(.78). 


10 


(71)' ■ 


n.s 


•lit' 


Strongly disagrees/disagrees ,' . 


19 


(53), 




(57) 


.01 


■ ' .5^ 


V, . ■ • ■ .■: ' . ' 

By taking part Irt' surveys 1 can affect>fche , ; 
government's decisions: . 














: ,' Strongly agrees/agrees . , . / . , , ./{ , , . 


11 


(82) 


n 


(15) 


n.s 


.25 


Strongly disagrees/disagrees ,'..,/' • • • 


16 


(50) ' 


* 


(55) 


• .01 





, ; ^The distribution for thos^ p agreed. strongly (N=22) was,as follows: Short Interview group (11=12,); 
long Interview. group (N=10),. All of these persons were wl J ^ : / 
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17 
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} ^ * ■ ■ 

ImpUcattons of the Study Results . 

' ' ■ : ■ • ■ . 

Tfie findings sunwarlzed above have a number;of theoretical and 
practical Implications for researchers and survey sponsors. 

K Although it would no doubt be appropriate to test other 
"effort'* treatments, the findings suggest that effort as ope rational! zed 
for this study (Ke., asking respondents to p/cjvlde estimates vs. asking 
them to check recojds) does not affect self-perceived respondent burden. 

2* Before entrance t'o the household is gained, the disclosed 
length of an interview does not appear to affect refusal* decisions. As 
suggester) by this and other current research, burden concerns do not 
appear 1:0 be the primary reason for most refusals. ^ 

3. However, based on our findings, the conventional wisdom about 
the burdensomeness of lengthy interviews finds some support when appl led ^ 
to those persons who h^ve agreed to be interviewed, at ' least for a 
Hgeneral interest" survey which does not deal with matters highly germane 
to the Interests of the respondent. (As noted earlier, the burcfensomeness 
levels for interviews dealing with highly respon^dent-pert inent issues, 
or for Interviews seeking opinions rather than factual information, may 
be very different.) • . .. 

Belief in the efficacy "of-surveys clearly emerged as an Impor- 

tant e1en>snt affecting feelings of burden, perhaps even overshadowing 
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actual interview length. The^indfngs suggest that it Is not a question 

so nwch^-oAthe importance of the speci f ic sXvey (jitsel f as a more general 

belief in the efficacy of surveys (or perhabs in the efficacy of indi vi-duals 

to affect the actions of decision-makers) wiich is Operative, It follows, 

therefore, that to reduce selvf ^perceived burden, i t i s important to convey 



to potential respondents the. importance and Usefulness of the survey method, 
and the likelihood that survfey data will be used by survey sponsors. 

5/ Finally, it is important to re-emphasize that the current 
research findings are based an a situation involving voluntary face- 
to-face household interviews. These "fi ndi ngs will hopefully be infor- 
mative to education researchers wh(3 conduct studies in sJmilar 's i tuat i ons . 
For example, our findings may be applicable to ef focts-'i rivoT^Tr^ personal 
Interviews with community members,,^pa refits, or student^. However, our 
findings may not necessarily apply 'to other contents in which education 
research is typically conducted—for example, to research involving , 
mandatory data col lection from school admi n i strators , or to research 
Involving "captive audiences" such as classroom teachers or^^support 
personnel. Measurement of respondent bur Jen under/these latter Conditions 
will require research specifically targetted 'to those contexts. 



